What is Supervised Learning?

• Learning from labelled examples to predict outcomes on new data.
• You provide features (X), the input variables describing each observation, and the target (y), the outcome you want to predict.
• The model learns the relationship between X and y, and predicts the value of y given X.

Two Types of Supervised Learning

• Classification - Predicting categories
  • Will this patient be readmitted? (Yes/No)
  • Which diagnosis code applies? (A, B, C, or D)
  • Binary or multi-class outcomes
• Regression - Predicting continuous numbers
  • How long will this patient stay in hospital? (days)
  • What will next month’s A&E attendances be? (count)
  • Numerical outcomes

Classification vs Regression

Note - Some algorithms work for both (e.g., decision trees)

The Generalisation Challenge

• The goal of supervised learning is to learn patterns that work on new data, not just training data.
• But this is difficult. Models can underfit and overfit, and will not generalise well.
  • Underfitting - Model too simple, misses patterns, performs poorly on training and test data.
  • Overfitting - Model too complex, memorises training data, performs well on training but poorly on test data.
• The sweet spot is a model that captures real patterns/signal but ignores noise.

The Learning Process

• Models learn by optimising a mathematical objective during training.
• This involves finding patterns that minimise the prediction error in training.
• What happens during .fit():
  1. Model makes predictions on training data
  2. Calculates how wrong those predictions are
  3. Adjusts internal parameters to reduce errors
  4. Repeats until errors stop decreasing

No Free Lunch Theorem

• Different algorithms use different strategies to find patterns and optimise the mathematical objective.
• Some algorithms deal well with simplicity, others work well with sparse data, some can handle incredibly complexity.
• But there is no single “best” algorithm.
  • This is known as the No Free Lunch Theorem.

Algorithms vs Models

• The algorithm is the learning approach (the recipe).
  • Logistic regression, decision tree, random forest
• The model is the trained result (applying the recipe to your data).
  • A random forest algorithm trained on Titanic data becomes a model
• You can use many different algorithms, but the workflow remains the same.

Many Algorithms to Choose From

• Linear methods
  • Linear Regression, Logistic Regression, Ridge, Lasso
• Tree-based methods
  • Decision Trees, Random Forests, Gradient Boosting (XGBoost, LightGBM, CatBoost)
• And many other approaches
  • K-Nearest Neighbours, Support Vector Machines, Neural Networks

The Workflow is Plug & Play

# swap algorithms, keep the workflow
clf = LogisticRegression()      # or
clf = DecisionTreeClassifier()  # or
clf = RandomForestClassifier()  # or
clf = GradientBoostingClassifier()

# same workflow regardless
clf.fit(X_train, y_train)
predictions = clf.predict(X_test)
accuracy = accuracy_score(y_test, predictions)

Choosing the Right Model

• Consider interpretability:
  • Model/predictions needs to be easy to explain - Simpler models
  • Predictive performance matters most - Complex models
• Consider your data:
  • Small datasets (< 1000 rows) - Simpler models, less overfitting risk
  • Large datasets - Complex models can find subtle patterns
• Consider constraints:
  • Limited compute - Faster algorithms
  • Deploying to production - Consider training/prediction time
• Best practice - Try multiple models, compare on hold-out (testing) data

Thank You!

Contact:

Code & Slides:

/NHS-South-Central-and-West/code-club

… And don’t forget to give us your feedback.